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Abstract 


Consider  an  autoregressive-moving  average  (ARMA)  discrete-time 

sequence  {x^}  with  covariance  sequence  {R^}.  Equations  are  given 

for  the  solution  of  the  AR  coefficients  {a,  }n  in  terms  of  the  covariance 

k 1 

{RrIo*1  \ and  subsequent  solution  for  the  MA  coefficients  fb^}j  in 
terms  of  the  AR  coefficients  and  the  covariances  {R^}^  ^ . The  results 
are  derived  and  presented  somewhat  differently  than  usual  to  complement 
the  results  of  [l]  for  the  synthesis  of  covariance-invariant  digital 
filters.  In  the  context  of  spectrum  analysis,  the  results  provide  a 
means  of  performing  ARMA  spectrum  analysis  on  data  that  arise  as 


sampled  data  from  a rational  continuous-time  process  [2].  An  important 
result,  originally  derived  in  [2],  shows  that  the  ARMA  spectrum  can  be 
obtained  without  actually  solving  the  nonlinear  f actorization  problem 
for  the  MA  coefficients. 


.1 


J 


I.  Introduction 


Consider  a zero-mean  wide-sense  stationary  process  x(t)  generated 
as  the  output  of  an  nth-order  continuous-time  filter  excited  by 

white  noise  and  with  covariance  function  R (t).  A discrete-time 

c 

00 

sequence  {x^)q  with  covariance  sequence  {R^}  is  said  to'  be  covariance- 
invariant  with  x ( t ) if  = Rc(t=kT)  for  k - 0,  +1,  +2,  ...  . The 
sequence  {x^)  may  be  obtained  as  the  output  of  an  ARMA  (n,n-l)  discrete- 
time filter  H(z) . H(z)  is  thus  said  to  be  covat iance- invar iant  with 
Hc(s)  [l],  [2]. 


Covariance-invariant  filters  arise  in  a number  of  contexts,  including 

signal  processing  applications  [l| , ARMA  spectrum  estimation  [2]  , and 

speech  analysis.  The  problem  is  to  synthesize  the  discrete-time  filter 

H(z)  from  knowledge  only  of  the  {R^}^11  Wien  synthesizing  covariance- 

invariant  digital  filters,  the  ^ are  characterized  by 

R,  - R (r=kT)  with  R (x)  obtained  from  a given  transfer  function  H (s) 

K c c c 

as  outlined  in  [ij , [2],  or  [lo] . In  spectrum  and  speech  analysis 
applications,  the  {R  ^ are  replaced  by  estimated  covariances  as 

discussed  in  [ 2 l . 


1 1 . Some  Basic  Results  for  Discrete-Time  Systems 


Every  nth-order  continuous-time  filter  H (s)  has  a covariance- 

c 

invariant  H(z)  that  is  ARMA  ( n , n — 1 ) [l],  [ 2]  - Therefore,  it  suffices 
to  consider  transfer  functions  of  the  form 
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b.z  ^+b„z  ^+. . .+b  z 

H(z)-  + -I  + -5— » ao = 1 

aA+a.z  +. . .+a  z 
0 1 n 


This  is  the  transfer  function  of  an  ARMA  (n,n-l)  discrete-time  filter. 

It  is  termed  strictly  proper  because  lim  H(z)  = 0.  The  filter  zH(z) 

Z-KO 

has  the  same  spectrum  as  H(z),  but  is  only  proper;  i.e.,  lim  zH(z3  <°°. 

7~*co 

Consideration  of  strictly  proper  transfer  functions  simplifies  the 
state-space  model  of  interest  to  us,  so  that  we  assume  a transfer  function 
of  the  form  (1)  throughout. 


The  difference  equation  corresponding  to  (1)  is 


% Vk-t 


“k-« 


where  { } and  (x^)  are,  respectively,  input  and  output  sequences, 
unit  pulse  response  of  H(z)  is  {h^}  where: 


bk  “ 1 ai\-a  t k>0 


The  companion  form  state  model  for  (2)  is  [a] 


Vl  = ♦ *k  + hUk 


Y ; Y = (1  0. . . 0) 


p 1 i 

|^-ara2...-anJ 


h = (h... .h  ) 
1 n 


0 an  (n-l)xl  vector  of  zeros  and  1 an  (n-l)x(n-l)  identify  matrix. 
Initial  conditions  follow  from  (3)  and  prime  denotes  transpose. 
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The  characteristic  equation  for  $ is 


An  + a,An-1  + ...  + a A + a = 0 
1 n-1  n 


The  Cayley-llamil  ton  theorem  says 
equation : 


satisfies  its  own  characteristic 


n 


Z a «n-J'  = 0 
i=0 


a0  = 1 


III.  Covariance  Results 


Now  assume  {u^l  is  a unit-variance,  zero-mean  white  sequence. 


For  $ a stability  matrix,  the  steady-state  covariance  of  {x^}q  is 


simply 


\ ■ <»  Vn  • K - U 2, 


where  (M)  denotes  the  (1,1)  element  of  M.  The  matrix  V,  is  the 
11  d 


steady-state  covariance  of  the  state  4 : = $V  $ hi/. 


For  k ^ n,  we  may  write 


I a K,  = (<ik  n E a d>n~m  V,)n 
m=0  m Vm  m=0  m d 11 


k > n 


But  from  the  Cayley-Hamillon  theorem,  the  RHS  of  (8)  is  zero. 
Therefore, 


l a = 0 


m=0 


m K-m 


k > n 


That  is,  for  covariance  lags  greater  than  or  equal  to  n,  an  ARMA 
(n,n-l)  system  "looks  purely  autoregressive."  | 2j  , [5],  | 6 | , f 7] , 


(5) 


(6) 


(7) 


(8) 


(9) 
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.a> 

An  alternative  expression  for  the  covariance  sequence  IR,  1S 


R.  = R . = E h h k = 0,  1,  2,  ... 
k -k  . m m+k 

m=l 


(10) 


Substitution  of  (3)  for  h and  a modest  amount  of  algebra  leads  to 

m+k 

the  following  result: 


n 


l 

£=0 


n-k 

L h b 
. m m+k 
m=l 


k = 0,  1,  2,  ... 


(ID 


For  k >_  n this  gives  the  same  result  as  (9).  Note  also  that  when 

2 

b = 0 for  m > 1 (an  AR  system),  the  RHS  of  (11)  is  b . Thus  (11)  is 
m 1 

a general  set  of  normal  equations  that  includes  the  AR  normal  equations 
as  a special  case. 

IV . The  Moving  Average  Coefficients 
To  obtain  an  expression  for  the  MA  coefficients  (b^)^  in  terms  of 
the  AR  coefficients  (a  }”  and  the  covariances  \ <;e  write  out 

(11)  for  k = n-1,  n-2,  ...,  1,  0,  use  the  result  of  (3)  for  h^,  and 
substitute  from  top  to  bottom.  We  thus  obtain  the  following  system  of 
nonlinear  equations  in  the  MA  coefficients 


n-k 

L bibS.+k  = rk 
i=l 


k = 0,  1 , . . . , n-1 


r 


k 


n-k-1 

I 

m=0 


1 ^k+m- 1 


(12) 
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The  expression  for  the  {r^}  may  alternatively  be  writti 


r = E a E a R.  . 
k _ m £ K+m-S, 

m=0  £= 0 


This  is  the  expression  one  obtains  for  the  RHS  of  (12)  by  proceeding 

n-k 

directly  from  (2)  to  derive  an  expression  for  E b b , Equations 

£,=  1 

(12)  and  (13)  are  entirely  equivalent.  The  extra  terms  in  (13)  vanish 


by  virtue  of  the  linear  dependence  established  in  (9). 


There  is  a spectral  factorization  problem  imbedded  in  the 


equations  of  (12).  Write 


1 b £b £+k  = rk  ’ k = °*  i1'  ±2>  •• 


with  b^  = 0 for  £ <_  0 and  £ >_  n+1,  r^  = 0 for  ]k]  ^ n,  and  r^  = r 


Note  {r  } is  simply  the  covariance  of  the  residual  sequence  { E ax 
k £=0 


Z-transform  (14)  to  obtain 
B(z)B(z_1)  = S (z) 


n , n _ 

B(z)  = E b z~  = z"n  E b zn_1C 
k=l  K k=l 


n- 1 

S (z)  = E 


1 rkz"k  = ro  + 1 rk  {\  + zk} 

k=-(n-l)  * U k-1  k k 


When  the  spectrum  S(z)  is  factored  for  B(z),  a polynomial  of  the 


following  form  is  obtained 


B(z)  = Kz  n (z-z,  ) 
k-1 


A 
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But  (B(z)B(z  1)  is  a mirror  image  polynomial,  as  the  result  S(z)  = S(z 
shows.  Therefore,  one  may  as  well  use  for  B(z)  the  expression  of  (17) 
with  an  arbitrary  number  of  the  z^  replaced  by  z^.  Thus  there  are 
M = 2n  1 different  solutions  for  B(z)  and,  correspondingly,  2°  1 different 
sets  of  MA  coefficients  one  may  choose.  The  unique  set  for  which  B(z)  has 
all  its  zeros  inside  the  unit  circle  is  the  minimum  phase  set.  There  may 
be  other  sets,  such  as  the  set  with  all  zeros  outside  the  unit  circle,  that 
have  more  desirable  "phase  properties." 

V . Application  to  Covariance-Invariant  Digital  Filter  Design 

The  preceding  results  are  applied  to  the  synthesis  of  covariance- 
invariant  digital  filters  as  follows: 

(1)  Begin  with  a stable  continuous-time  transfer  function  H^Cs) 
and  solve  for  the  corresponding  covariance  function  R^(t). 

See  £l"j,  |~2],  and  fio]  for  useful  expressions.  Set  = Rc(i=kT). 

(2)  Write  out  (9)  for  k=n,  n+1,  ...,  2n-l  and  solve  the  resulting 
Toeplitz  set  of  linear  equations  for  the  AR  parameters  {a^}^. 

See  £7]  and  for  references  to  related  equations  and 

software  for  efficiently  solving  such  equations.  See  [7J  for 
a more  general  discussion  of  the  related  "normal  equations" 
that  arise  in  autoregressive  or  linear  prediction  of  speech. 

(3)  Solve  either  (12)  or  (15)  for  the  MA  coefficients  {b^}*^.  If 
(12)  is  solved,  then  a polynomial  B(z)  may  be  formed  from  the 
{b^}"  and  factored  for  its  roots.  Roots  lying  outside  the 
unit  circle  may  be  reflected  inside  to  obtain  a variety  of 
phase  functions,  ranging  from  minimum  to  maximum  phase. 
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VI . Spectrum  Analysis 

Consider  the  power  spectrum  S (f)  corresponding  to  the  filter  H(z)  : 

d 

q u/  j2irfT.„,  -1  -j  2tt  fT. 

S (f)  = H(z=eJ  )H(z  =e  J ) 
d 


n n 
Z l b b e 
= k=l  £=1  K 


j2n(£-k) fT 


n n 

Z Z a.  afte 
k=0  <1=0  k * 


j2it()l-k)  fT 


(18) 


The  double  sums  in  the  numerator  and  denominator  may  be  evaluated  on 

upper  and  lower  triangular  grids  to  write 
n n-1  n-i 

* bk+2  Z 1 bkbk+£  COS  2nifT 

sd(f)  = izlJsd- - 


n 


n n-x 


(19) 


Z a.  4*  2 Z a cos  27TkfT  4 2 Z Z a.  a cos  2~£fT 

k=0  k k=l  k <1=1  k=l  k k+* 


From  (12)  we  may  write 
n-1 


V£)  ■ 


rn  + 2 Z r cos  27r£.fT 
u 1=1  1 


(20) 


n „ n n n-SL 

I a,  + 2 Z a,  cos  2rkfT  +2  Z Z a,  a cos  2rP,  fT 

k=o  k k=i  k e=i  k=i  K k+f' 


This  result  shows  that  the  ARMA  spectrum  may  be  found  in  terms  of  the  AR 
parameters  {a.,}^,  and  the  residual  covariance  {r^}^  \ without  ever  solving 
for  the  MA  coefficients.  Thus,  a nonlinear  spectral  factorization  problem 


is  solved. 
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VII . Concluding  Remarks 


When  the  results  discussed  here  are  used  for  spectrum  analysis,  then 


the  covariance 
, 2n-l 


{V 


2n-l 

0 


are  replaced  everywhere  by  estimated  variables 


. Gersch  [6^  has  shown  that  when  the  order  n is  known,  and  co- 
- _1  N 

variances  are  estimated  as  = N E x?,xjl+k  ’ t^ien  ttie  AR  coefficients 


found  from  the  normal  equations  are  unbiased  and  consistent.  It  does  not 
follow,  however,  that  for  finite  N the  AR  coefficients  will  correspond  to 
a stable  filter  H(z).  That  is,  the  equations  of  (9)  do  not  enjoy  the  nice 
stability  property  associated  with  the  normal  equations 


n 

E a R_  = 0 , k = 1,  2,  ...,n 
« • m R-m 
m=0 

This  defect  may  not  be  critical  from  the  point  of  view  of  spectrum  analysis. 

If  a stable  filter  is  desired,  unstable  poles  of  H(z)  may  be  reflected  inside 
the  unit  circle.  A more  substantial  defect  is  that  the  estimated  residual 
sequence  {r^l  may  not  be  a true  covariance  sequence.  This  means  the 
numerator  of  (20)  may  be  negative  for  some  f.  Thus,  care  must  be  taken 
when  using  (20)  to  ensure  that  a negative  spectrum  is  not  estimated.  This 
consideration  is  but  a subset  of  the  more  general  problem  of  order  determinntie 
in  AR  and  ARMA  spectrum  estimation.  A thorough  study  of  order  determination  in 
ARMA  (n,n-l)  spectrum  analysis  is  required  before  such  analysis  becomes  a 
well  developed  tool.  Nonetheless,  we  feel  ARMA  (n,n-l)  spectrum  analysis 
is  the  natural  approach  when  dealing  with  data  that  arise  as  sampled  data  fr.  rt 
a rational  continuous-time  process.  The  calculations  are  not  markedly  more 
complicated  than  for  AR  (n)  spectrum  analysis. 

See  6~J  for  a more  complete  list  of  references  to  the  statistics 
literature  dealing  with  ARMA  time  series. 
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